Evaluation of catchment models

Catchment models are by definition simplified representations of the real world system. This aggregation takes place in space and time and has several important consequences. First, there are no generally applicable rules to perform this aggregation, and the resulting model structure is usually a function of the modeller’s hydrological understanding. Secondly, the model parameters cannot be measured directly in many cases, but have to be estimated. In this process, we usually assume that the parameters are constant in time and representative of inherent properties of the real system. This is particularly relevant when transferring parameters to ungauged catchments. The modeller’s task is to find a model, i.e. a combination of model structure and parameter set(s), suitable for the anticipated modelling purpose, data and catchment characteristics. Traditionally, the modeller defines an objective function, i.e. some aggregated measure of the distance between simulated and observed system response, and minimizes (or maximizes, depending on definition) its value, a procedure usually called calibration. The aim is to match simulated and observed system behaviour. This is often followed by an application of the identified model to another part of the time series not used during calibration, to show that it can be applied generally. This is usually called the validation step. Many studies have shown that this type of approach is insufficient to test adequately the suitability of a model and that the scientific conclusions that can be drawn from such a procedure are very limited. A commonly found result is that several, often very different, parameter sets and even model structures are equally acceptable system representations in this context (e.g. Beven and Freer, 2001). Therefore, we seek to apply approaches to model evaluation that are more discriminative. There are at least three dimensions in which this evaluation should be performed (Figure 1): (1) performance; (2) uncertainty; and (3) ‘realism’.